Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix_: restore node config #6270

Draft
wants to merge 1 commit into
base: fix/v1_upgrading
Choose a base branch
from

Conversation

qfrank
Copy link
Contributor

@qfrank qfrank commented Jan 20, 2025

Caution

This PR is not intended to be merged

This is a continue work based on PR, the fix could fix the error:

ERROR[12-23|23:54:56.136|github.com/status-im/status-go/mobile/status.go:474]     loginAccount failed                      error="failed to start node: Key: 'NodeConfig.NetworkID' Error:Field validation for 'NetworkID' failed on the 'required' tag\nKey: 'NodeConfig.LogLevel' Error:Field validation for 'LogLevel' failed on the 'eq=ERROR|eq=WARN|eq=INFO|eq=DEBUG|eq=TRACE' tag"

the solution is:

  • pass relate node config params that only mobile frontend configured/knows when invoke loginAccount
  • when loginAccount get invoked, check if there is one bad(e.g. networkid is 0 and networkid is not 0 in settings.node_config) node config record on status-go side, if yes, create a default node config for the user, we also need take care of existing node config values in settings.node_config when combining the default config and existing one

however, this solution will make loginAccount looks not beautiful anymore.

following is the opinion from @ilmotta

We could try to create a special build release build for this user without merging PRs, just having the status-go and mobile branches would be enough. That would make our lives easier because we'd be able to skip QA and we wouldn't need to worry too much about loginAccount getting ugly.
But if the problem is solved, then a second step could be to make the PRs proper and see if we can get them merged. Probably then, the adaptions/fixes you found to work with 3esmit would help other users who never reported anything to us.

Relate mobile PR
Relate comment

@status-im-auto
Copy link
Member

status-im-auto commented Jan 20, 2025

Jenkins Builds

Click to see older builds (11)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 77e75a4 #1 2025-01-20 01:16:51 ~3 min macos 📦zip
✔️ 77e75a4 #1 2025-01-20 01:17:42 ~4 min windows 📦zip
✔️ 77e75a4 #1 2025-01-20 01:17:55 ~4 min linux 📦zip
✔️ 77e75a4 #1 2025-01-20 01:18:45 ~5 min macos 📦zip
✖️ 77e75a4 #1 2025-01-20 06:51:26 ~5 hr 38 min tests 📄log
✔️ 77e75a4 #1 2025-01-20 06:52:47 ~5 hr 39 min android 📦aar
✔️ 77e75a4 #1 2025-01-20 06:53:37 ~5 hr 40 min tests-rpc 📄log
✔️ 8a439b4 #2 2025-01-20 14:24:19 ~2 min windows 📦zip
✔️ 8a439b4 #2 2025-01-20 14:24:47 ~3 min macos 📦zip
✔️ 8a439b4 #2 2025-01-20 14:25:54 ~4 min linux 📦zip
✔️ 8a439b4 #2 2025-01-20 14:26:53 ~5 min macos 📦zip
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ b0b61de #3 2025-01-21 11:06:20 ~3 min windows 📦zip
✔️ b0b61de #2 2025-01-21 11:06:52 ~3 min ios 📦zip
✔️ b0b61de #3 2025-01-21 11:08:23 ~5 min macos 📦zip
✔️ b0b61de #3 2025-01-21 11:08:30 ~5 min linux 📦zip
✔️ b0b61de #3 2025-01-21 11:08:39 ~5 min android 📦aar
✔️ b0b61de #3 2025-01-21 11:08:44 ~5 min macos 📦zip
✔️ b0b61de #3 2025-01-21 11:09:48 ~6 min tests-rpc 📄log
✔️ b0b61de #3 2025-01-21 11:34:18 ~31 min tests 📄log
✔️ d7f725f #4 2025-01-21 11:09:24 ~2 min windows 📦zip
✔️ d7f725f #3 2025-01-21 11:11:24 ~4 min ios 📦zip
✔️ d7f725f #4 2025-01-21 11:13:34 ~5 min macos 📦zip
✔️ d7f725f #4 2025-01-21 11:13:48 ~5 min linux 📦zip
✔️ d7f725f #4 2025-01-21 11:14:16 ~5 min macos 📦zip
✔️ d7f725f #4 2025-01-21 11:15:18 ~6 min android 📦aar
✔️ d7f725f #4 2025-01-21 11:16:15 ~6 min tests-rpc 📄log
✔️ d7f725f #4 2025-01-21 12:06:03 ~31 min tests 📄log

@qfrank qfrank force-pushed the fix/node_config_migration branch from 77e75a4 to 8a439b4 Compare January 20, 2025 14:21
@jakubgs
Copy link
Member

jakubgs commented Jan 21, 2025

You will need to rebase your branch on develop in order for the CI job to run using new Nix interpreter:

@qfrank qfrank force-pushed the fix/node_config_migration branch from 8a439b4 to b0b61de Compare January 21, 2025 11:02
@qfrank qfrank requested a review from a team as a code owner January 21, 2025 11:02
@qfrank qfrank force-pushed the fix/node_config_migration branch from b0b61de to d7f725f Compare January 21, 2025 11:06
Copy link

codecov bot commented Jan 21, 2025

Codecov Report

Attention: Patch coverage is 80.00000% with 13 lines in your changes missing coverage. Please review.

Project coverage is 61.77%. Comparing base (45d97be) to head (d7f725f).

Files with missing lines Patch % Lines
api/geth_backend.go 80.00% 7 Missing and 6 partials ⚠️
Additional details and impacted files
@@                 Coverage Diff                  @@
##           fix/v1_upgrading    #6270      +/-   ##
====================================================
- Coverage             61.79%   61.77%   -0.02%     
====================================================
  Files                   845      845              
  Lines                111336   111399      +63     
====================================================
+ Hits                  68797    68819      +22     
- Misses                34572    34601      +29     
- Partials               7967     7979      +12     
Flag Coverage Δ
functional 21.49% <0.00%> (+0.01%) ⬆️
unit 60.32% <80.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
api/defaults.go 81.40% <ø> (-0.16%) ⬇️
protocol/requests/login.go 57.14% <ø> (ø)
api/geth_backend.go 55.23% <80.00%> (+0.99%) ⬆️

... and 30 files with indirect coverage changes

@yakimant
Copy link
Member

hm, not sure I an help with review

@qfrank qfrank removed the request for review from a team January 22, 2025 09:02
Copy link
Collaborator

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's for a single user, and it won't be merged, I don't think we needs code review.
If it works for the user - great, if it doesn't - then there are issues in the fix 😄

I have added do not merge labels and a warning in description.
@qfrank, perhaps makes sense to convert this PR to draft as well?

--

But if the problem is solved, then a second step could be to make the PRs proper and see if we can get them merged. Probably then, the adaptions/fixes you found to work with 3esmit would help other users who never reported anything to us.

@ilmotta, do you think we need to persuade such little issues for few users?
This could even be done manually amending the DB 🤷‍♂️

Comment on lines +601 to +604
if currentConf.NetworkID == 0 &&
currentConf.KeyStoreDir == "" &&
currentConf.DataDir == "" &&
currentConf.NodeKey == "" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the problem is that the node_config is empty in the database, but... why is this a problem? Can't we just use the default values, without writing them to the database?

Perhaps this is also related (and could be mitigated by) #5597. It seems that loadNodeConfig actually gives priority to DB values rather than the values passed with CreateAccount/LoginAccount/.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the problem is that the node_config is empty in the database, but... why is this a problem? Can't we just use the default values, without writing them to the database?

default values with backend are not enough, we don't have following values:

  • verifyTransactionURL
  • verifyENSURL
  • verifyENSContractAddress
  • verifyTransactionChainID
    They're needed to pass from frontend currently.

Perhaps this is also related (and could be mitigated by) #5597.

it's not related :)

It seems that loadNodeConfig actually gives priority to DB values rather than the values passed with CreateAccount/LoginAccount

loadNodeConfig support passing inputNodeCfg which will override values from DB

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default values with backend are not enough, we don't have following values:

  • verifyTransactionURL
  • verifyENSURL
  • verifyENSContractAddress
  • verifyTransactionChainID
  • They're needed to pass from frontend currently.

@friofry will we need these parameters to be passed from the client after merging #6178 ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised that these fields are related :) Initially I wanted to answer that my pr is related to another table in the database. But then I found this line working as a fallback:

nodeConfig.ShhextConfig.VerifyENSURL = mainnet(request.WalletSecretsConfig.StatusProxyStageName).FallbackURL

And this field will be removed soon, as well as Status Proxy will eventually be replaced by Smart Proxy. Which will always be in-memory and won't depend on the database.

also loadNodeConfig is really fragile and complex. We probably need to collect all the issues and rework them all.

@qfrank
Copy link
Contributor Author

qfrank commented Jan 24, 2025

Since it's for a single user, and it won't be merged, I don't think we need code review.

Sorry for confusing, let me explain further.

If old user didn't upgrade to v1.19.0 first rather than upgrade to v2+ directly, they won't be able to login (relate comment), so I'm afraid this bug could affect many old users... it's just because only 3esmit reported the error that attracted our attention. We have chosen to focus on and actively try to solve the compatibility of v2 with v1 in the past, but due to various bugs and the lack of attention to compatibility during development for a period of time, we failed. If you accept this not-so-pretty solution, maybe we can consider merging this PR, or @igor-sirotin if you have a better solution you can also propose it if you would like to.

A better solution might be to migrate configuration values like verifyXXX (e.g. verifyTransactionURL) to the backend without modifying the frontend code.

However, speaking of which, even if we merge this PR, how much value does it bring, because users like 3esmit who can persist to continue using status should be a minority. cc @ilmotta

@qfrank qfrank marked this pull request as draft January 24, 2025 11:55
@ilmotta
Copy link
Contributor

ilmotta commented Jan 24, 2025

@ilmotta, do you think we need to persuade such little issues for few users?

As @qfrank mentioned above (#6270 (comment)) we believe there is or was a large pool of users who were locked out, and perhaps many of them never came back after the frustration of losing their local data. Could very much be related to our historical drop in users over time.

@igor-sirotin our original intention in the mobile PR and this one in status-go is to only proceed to merge if the solution resolves the users' bug AND is of course deemed mergeable by reviewers. The user hasn't yet upgraded his prod build from App Store. We need to provide this build to him.

I guess this PR for now is just to see if it sort of makes sense in everybody's head as a pragmatic solution.

This could even be done manually amending the DB 🤷‍♂️

The problem is how does the user modify the DB from a production build in their phone? For instance, in iOS the user was able to somehow backup the entire phone and access the Status data directory, decrypt the DBs and attempted to modify the DB in hopes of fixing the dirty DB, but to no avail yet because it's hard for the user and us to know exactly what to modify. We also can't fix directly for the user because it's their data. In Android things can be worse because if the phone is not rooted, which is the majority of cases, the user can't access the files in the data folder. And if they try to unlock the bootloader the phone data is going to be wiped out.


Next week or sometime in the near future I'd love if we can start to analyze the problem of users getting locked out due Status bugs. I'll think more carefully about all this and reach out to you guys. In particular mobile users, because their situation is hopeless without a form of exporting and importing data due to the way most mobile OSes prevent data access. status-im/status-mobile#20996 (comment) In desktop OSes users already have the freedom to access and modify their data however they see fit, so it's much nicer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants